Training Restricted Boltzmann Machines via the Thouless-Anderson-Palmer Free Energy
نویسندگان
چکیده
Marylou Gabrié, Eric W. Tramel and Florent Krzakala 1 Laboratoire de Physique Statistique, UMR 8550 CNRS, Department of Physics, École Normale Supérieure and PSL Research University, Rue Lhomond, 75005 Paris, France 2 International Centre for Fundamental Physics and its interfaces at Ecole normale suprieure, 75005 Paris, France 3 Sorbonne Universits, UPMC Univ Paris 06, UMR 8550, LPS, F-75005, Paris, France (Dated: June 16, 2015)
منابع مشابه
Training Restricted Boltzmann Machine via the Thouless-Anderson-Palmer free energy
Restricted Boltzmann machines are undirected neural networks which have been shown to be effective in many applications, including serving as initializations for training deep multi-layer neural networks. One of the main reasons for their success is the existence of efficient and practical stochastic algorithms, such as contrastive divergence, for unsupervised training. We propose an alternativ...
متن کاملPerturbative Interpretation of Adaptive Thouless-Anderson-Palmer Free Energy
In conventional well-known derivation methods for the adaptive Thouless-Anderson-Palmer (TAP) free energy, special assumptions that are difficult to mathematically justify except in some mean-field models, must be made. Here, we present a new adaptive TAP free energy derivation method. Using this derivation technique, without any special assumptions, the adaptive TAP free energy can be simply o...
متن کاملA Deterministic and Generalized Framework for Unsupervised Learning with Restricted Boltzmann Machines
Restricted Boltzmann machines (RBMs) are energy-based neural-networks which are commonly used as the building blocks for deep architectures neural architectures. In this work, we derive a deterministic framework for the training, evaluation, and use of RBMs based upon the ThoulessAnderson-Palmer (TAP) mean-field approximation of widely-connected systems with weak interactions coming from spin-g...
متن کاملInformation Geometry of Mean-Field Approximation
I present a general theory of mean-field approximation based on information geometry and applicable not only to Boltzmann machines but also to wider classes of statistical models. Using perturbation expansion of the Kullback divergence (or Plefka expansion in statistical physics), a formulation of mean-field approximation of general orders is derived. It includes in a natural way the "naive" me...
متن کاملThouless-Anderson-Palmer Approach to the Spherical p-Spin Spin Glass Model
We analyze the Thouless-Anderson-Palmer (TAP) approach to the spherical
متن کامل